The delivery of graphical information over narrow bandwidth network connections often requires some compromise between network bandwidth, CPU resource usage, and image quality. There are multiple different image encoding schemes, or “formats,” available, using various image compression technologies. Some image compression technologies are more appropriate for certain types of images than others. Therefore, it can be important to select a preferred image encoding scheme and compression technology for the particular image being transmitted in order to minimize bandwidth while preserving acceptable image quality.
Display remoting technologies exist whereby a user may view the graphical user interface (GUI) of a computer from a location remote from the computer. For example, Virtual Network Computing (VNC) is graphical user interface remoting standard that uses Remote Frame Buffer (RFB) protocol to allow a client to remotely control a computer system over a persistent TCP connection, typically using TCP port 5900. The RFB protocol allows a server to update the frame buffer displayed on a VNC viewer running on a client machine. In general terms, a frame buffer typically occupies a portion of RAM used for temporary storage of image data that is available for display. In the RFB protocol, the server sends image buffer data corresponding to small rectangles of the graphical user interface (GUI) to the client, which the client then assembles to reconstruct the GUI at the remote location. Some VNC implementations, such as RealVNC, available from RealVNC Ltd., of Cambridge, UK, use an HTTP server to provide a VNC viewer to the client as a Java applet, which then connects to the server for remote UI over a separate persistent TCP connection. Yet another VNC implementation, referred to as “ThinVNC,” available from CybeleSoftware, Inc. of Wilmington, Del., uses the WebSocket protocol of HTML5 for remote access to a user interface. Another example of a technology that relies on HTML4 technology for display remoting is described in U.S. patent application Ser. No. 13/162,365, filed Jun. 16, 2011, which is incorporated herein by reference in its entirety.
One problem associated with remoting GUIs is the selection of an appropriate image format and corresponding compression method. The amount of compression that can be achieved directly impacts the bandwidth consumed in graphical display remoting and plays a key role in the latency and maximum frame rate. Various image digital encoding schemes and/or compression methods, such as a Portable Network Graphics (PNG) and a Joint Photographic Experts Group (JPEG) image encoding, which uses lossless and lossy compression methods, respectively, have been developed and used for compressing an image. Moreover, both PNG and JPEG compression methods are available to HTML5 based client applications.
However, the compression methods employed by PNG and JPEG image formats have limitations, and there are some types of images that are able to compress better with one type of compression method over the other. For example, while the PNG format uses a lossless compression method such that the exact original data can be reconstructed from the compressed data, the PNG format is unable to match JPEG compression ratios for video and photographic data. The JPEG format, however, is lossy, and as a result, the reconstructed image will not precisely match the original. Specifically, the JPEG compression method can generate visible artifacts of the compression at moderate compression ratios that are particularly noticeable when compressing text and sharp vector-based graphics—image types that are highly compressible using the PNG format. Because GUIs often have a mix of different types of image data—some portions containing text or vector-based graphics while other portions contain photographic images or color gradients, a selection of an encoding scheme that is appropriate for one area of a display may be unsatisfactory—e.g., too little compression or image compression artifacts—at another portion of the display. Furthermore, the locations of each type of image data within the GUI are not predictable.